Ubiquitous video and persistent surveillance are trends in the security and intelligence community. Operators may view video over long time periods from multiple sources. Moreover, surveillance is increasingly used in many domains. Persistent surveillance in cities and other populated areas supports anti-terror efforts and law enforcement both in real-time monitoring and forensic analysis. Similarly, military applications are increasing the use of unmanned aerial vehicles (UAV) and other platforms for gathering real time intelligence, and using video products information platforms to enhance analysis of events and trends. However, currently operators typically view video sources, i.e., monitoring applications, independent of other information regarding the video sources, such as information provided by metadata. This other information is usually displayed on the periphery of the monitoring applications.
Current solutions combining video and other information include Great Circle Technologies Minerva™ and Merlin's AutoFMV Workflow™, which provide transparent overlays of geospatial intelligence data. This approach uses ortho-rectification of the video stream to determine the geospatial coverage of the video frames. However, ortho-rectification is a complex process and is typically only suited for downward looking video streams, for example, from airborne platforms such as a UAV.
Another current solution is Nokia Mobile's augmented reality application. While this research project shows virtual features overlain on video, the Nokia application is restricted to a particular mobile phone platform and does not have generic camera and geospatial application support. For example, the Nokia application is designed to work on a unified sensor, video, and visualization platform based on a mobile phone. The Nokia application is not able to operate with distributed video, mapping, and viewing components that may be connected across any available network without being tied to one device. Additionally, the Nokia application performs analysis of the actual video stream to determine the location of an object based on the offset from the known camera position and the results of the analysis, which is a time consuming, complex operation. Further, the Nokia application draws graphics directly onto the video display by using simple line graphics and text to represent objects, accordingly limiting the types of data that can be used to populate the overlay.